Data transmission by extracted or calculated identifying data

ABSTRACT

The present invention relates to data transmission, and more particularly relates to use of extracted or calculated identifying data as proxies in transmission. One claim recites a method including: receiving identifying data and information from a handheld device, the identifying data having been calculated from audible portions or video portions of first content utilizing at least a programmed electronic processor housed within the handheld device, in which the handheld device comprises a media player for rendering audio or video content to a user of the handheld device, and the information comprises capabilities associated with the handheld device; with reference to a database, determining additional content, enhanced or improved relative to the first content, that is associated with: i) the identifying data, and ii) the information; and providing metadata associated with the additional content to the handheld device. These techniques could be advantageously applied to cell phones. Of course, other combinations and claims are also provided.

RELATED APPLICATION DATA

This application is a continuation of U.S. patent application Ser. No.11/274,758, filed Nov. 14, 2005 (U.S. Pat. No. 7,545,951), which is acontinuation of U.S. patent application Ser. No. 09/504,239, filed Feb.15, 2000 (U.S. Pat. No. 6,965,682). The 09/504,239 application is acontinuation-in-part of application Ser. No. 09/491,534, filed Jan. 26,2000, which is a continuation-in-part of application Ser. No.09/473,396, filed Dec. 28, 1999 (U.S. Pat. No. 6,577,746). The09/504,239 application also claims the benefit of U.S. ProvisionalPatent Application No. 60/134,782, filed May 19, 1999, the disclosure ofwhich is attached as Appendix A. The Ser. No. 11/274,758 application isalso a continuation-in-part of copending application Ser. No.09/476,686, filed Dec. 30, 1999, entitled Watermark-Based Personal AudioAppliance. Each of the above U.S. patent documents is herebyincorporated by reference.

FIELD OF THE INVENTION

The present invention relates to data transmission, and moreparticularly relates to use of extracted or derived data as proxies intransmission.

SUMMARY OF THE INVENTION

As detailed in the assignee's prior applications, including 60/134,782,60/141,538, and Ser. No. 09/343,104, digital watermark technology hasnumerous applications beyond its traditional role of simplycommunicating copyright information. One futuristic view foresees thatall “content” should be watermarked, thereby enabling a great variety ofoperations and transactions whenever watermarked content is processed bydigital devices equipped with watermark recognition and readingtechnology. All physical media objects can thereby be inherently andpersistently digitally-enabled, permitting greatly simplified access tonetworks and execution of local and remote applications. The continuinggrowth of the Internet and beginnings of trends toward pervasivecomputing signal an opportunity to radically change the relationshipsbetween traditional media content and digital processing environments.

In this specification, content refers not just to electronic audio,image, and video files, but also includes the content aspects ofphysical objects and media, e.g., artwork, patterns, and labels onproduct packaging, concert tickets, etc.

In accordance with a preferred embodiment of the present invention, theprocessing of watermark data as pointer to shared resources is sometimesused in lieu of transmitting from point to point the object with whichit is associated, thereby gaining efficiencies in speed and bandwidth.

This and other features and advantages of the present invention will bemore readily apparent from the following detailed description, whichproceeds with reference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating a network environment in whichprinciples of the present invention may be employed.

FIG. 2 is a flow chart illustrating aspects of one embodiment of thepresent invention.

FIG. 3 is a flow chart illustrating aspects of another embodiment of thepresent invention.

FIG. 4 is a flow chart illustrating aspects of yet another embodiment ofthe present invention.

DETAILED DESCRIPTION

Referring to FIG. 1, consider an exemplary network 10 linking twodevices—a first device 12 associated with an originating user, and asecond device 14 associated with a recipient user. The first device 12is coupled to the network through a relatively low bandwidth channel,whereas the second device 14 is coupled to the network through arelatively high bandwidth channel. (For example, the first device may bean internet-capable cell phone having low resolution-, still imageonly-capture capabilities, providing a 9600 baud data channel, or it maybe a home PC, with an associated PC or digital single shot camera,coupled to the internet with a 28.8 kbps modem. The second device may bea computer coupled to the internet through a 1.45 megabit per second T−1line, a cable modem, etc.) The network 10 connecting the two devicesincludes various links—narrow bandwidth at some parts (e.g., 16), verybroadband at other (e.g., internet backbone 18), etc.

Assume the user of device 12 encounters a printed image, e.g., anadvertisement in a magazine, that may be of interest to the user ofdevice 12. Using an imaging device (e.g., a CMOS- or CCD-camera builtinto a cell phone, a flatbed scanner connected to a PC, etc.), device 12captures an image of the advertisement.

In prior art techniques, the image captured by device 12 would have beensent to device 14 over the network; the image received by the seconddevice would be exactly the image sent by the first device.

In accordance with one embodiment of the invention, device 14 receives abetter image than that sent from device 12. In one such embodiment,device 14 receives the image data captured by device 12. Device 14recognizes that the image includes a watermark hidden within the imagedata, and decodes same. The watermark payload includes an index by whicha copy of the image can be accessed from a server 20 on the internet orother storage medium. With this index, the second device 14 queries theserver 20, which returns the image corresponding to this watermark index(in this case, the advertisement) back to the second device 14. Theimage provided by the server can be higher resolution or pristine, i.e.,it has no artifacts left from scanning at device 12, etc. Such aprocedure is shown by the flowchart of FIG. 2.

The watermark payload identifying the sensed image can as long or asshort as the application requires. Typically, payloads of between 16 and64 bits are used, although this is not essential. Shorter payloads havethe advantage that they can be more robustly encoded while maintaining afixed degree of image quality; longer payloads offer a greater universeof identifiers with which the image can be labeled. Illustrativewatermarking technology is detailed in the assignee's U.S. Pat. No.5,862,260 and in U.S. Pat. No. 6,614,914. A great variety of otherwatermarking arrangements may be used, including those proposed in U.S.Pat. Nos. 5,930,369, 5,933,798, 5,664,018, 5,825,892, 5,940,429 and5,889,868.

In accordance with another embodiment of the invention (FIG. 3), thebandwidth bottleneck imposed by narrowband channel 16 (through whichdevice 12 is coupled) is obviated by employing a watermark as a proxyfor an image. In such an arrangement, the image data captured by device12 is decoded, and a watermark payload hidden in the image is extracted.(This can be performed by hardware or software available in device 12,e.g., a cell phone microprocessor, a desktop computer, dedicated decodercircuitry, etc. Alternatively, this decoding can be done remotely fromdevice 12, but before device 14, e.g., by a smart router in theintervening network. In the following discussion, decoding in the device12 is assumed.) Instead of transmitting the image data over the network,the watermark decoding device (e.g., device 12) simply transmits thewatermark payload (or a part thereof). On receipt of the payload, device14 again queries the server 20, and obtains the image (and/or additionalcontent or functionality, as detailed below), corresponding to thatwatermark. The image is obtained over the high-speed channel(s) betweenthe server and the second device; the low bandwidth channel linking thefirst device conveys just the low bandwidth watermark payloadinformation.

By building filters into the low bandwidth devices, upon recognition ofa class of watermarks indicating availability of the image as a sharedresource, or upon user selection of “transmit only watermark data”, theimage [or content associated with it via the watermark] can be madeavailable to the message recipient via more capable transmission means.

A variant of the foregoing does not transmit the watermark payload tothe second device 14. Instead, the payload is dispatched by the firstdevice 12 (or the smart router) directly to the server 20, withinstructions that the corresponding desired image be sent to the seconddevice 14. Such an arrangement is shown in FIG. 4.

In some applications, the media delivered by the server may be richerthan the simple image captured by device 12. For example, the watermarkpayload in the image captured by device 12 may index one or more fileson server 20 that includes video, animation, sound, executableapplications, aplets (e.g., JAVA, ActiveX) etc (“enhanced content”).Thus, scanning of a magazine ad at one device can prompt delivery of avideo, a Macromedia ShockWave presentation, etc., to the second device.

In some embodiments, the second device 14 identifies to the server 20its media-playback capabilities. The server 20 can then respond to awatermark-based query with media appropriate to that particular mediaconsumer.

One way the media capabilities of device 14 can be indicated to server20 is by a data word comprising flag bits, with each set “1” bitindicating a capability. A simplified 8-bit capability word may be asfollows:

Bit Capability 0 GIF file display 1 TIFF file display 2 JPEG fileddisplay 3 AVI movie display 4 WAV sound 5 RealAudio sound 6 MP3 sound 7WindowsMediaThe data comprising this word may be automatically compiled on device14, e.g., from the operating system database with which programs areregistered on installation (the Registry database in Windows).

If device 14 sends the capability word 10101100 to server 20, the serverknows the device 14 supports GIF and JPEG imagery (but not TIFF), andRealAudio and WAV sound (but not MP3 or WindowsMedia).

If server 20 has media content corresponding to the queried watermark inseveral supported formats, it can deliver certain ones according to apriority order (e.g., send JPEG if supported; else send GIF ifsupported; else send TIFF if supported).

If the server 20 only has media in a format not supported by the seconddevice 14 (e.g., TIFF in the foregoing example), the server may invoke aconversion routine to perform an on-the-fly conversion to a supportedmedia type (e.g., JPEG) prior to sending to the second device 14.

If the watermark index is provided by the second device 14 (rather thandirectly from the first device 12), the capability data word canaccompany the index.

If the watermark index is provided directly from the first device 12,the server can solicit from the second device 14 a data capability wordbefore responding to the query. Alternatively, the server can keep,on-file, a database detailing the media capabilities of all known mediaconsumers, and can tailor its query response according to such profile.(The second device 14 can be arranged to automatically inform server 20of updates to its capability, e.g., each time a new media playbackapplication is registered in the registry database.)

If the server 20 does not know, and cannot discern, the mediacapabilities of the second device 14, it can provide media in a defaultform that is most likely to be acceptable (e.g., JPEG, if the contentcaptured by the first device 12 is imagery).

From the foregoing description, it will be apparent that embodiments ofthe present invention provide various advantages over the prior art. Oneis the dispatch of high bandwidth enhanced content using a low bandwidthchannel. Another is the receipt of higher-quality data than thatoriginally captured. Another is delivering applications via lowbandwidth channels to recipients by capturing images or watermark datafrom media content that serve as proxies for the applications.

Having described and illustrated the principles of our invention withreference to a specific embodiment, it will be recognized that theprinciples thereof can be implemented in other, different, forms.

For example, while the invention has been described with reference toimages, the same principles are equally applicable to video and audio.

Similarly, while the foregoing description has made reference totransmitting the watermark, in many implementations only a part of thewatermark need be transmitted. (The watermark may include errorcorrecting information, or other data, not necessary to identify thecorresponding data on the server 20.)

Still further, while the detailed embodiment contemplated a still orvideo camera system for first device 12, much of the functionality ofsuch an image capture system isn't essential to the present invention.Instead, an input device that serves a simpler “watermark capture”function may be used instead. Such a device can omit, e.g., hardware orsoftware components associated with pixel interpolation (commonly usedto achieve a desired virtual resolution), formatting (e.g., to provideoutput in JPEG form), etc. Such components serve useful functions whenthe resulting imagery is to be displayed or printed, but aresuperfluous—or detrimental—when the image data is simply to be decodedto extract watermark data.

While the invention is illustrated with reference to steganographicwatermark technology for identifying the initial content (i.e., thatsensed by device 12), other technologies can alternatively be used.These include data glyphs, 1- and 2-D barcodes, magnetic ink, RF IDtags, UV or IR markings, etc.

While the detailed embodiment contemplated a single server 20 to serveas the repository of content corresponding to watermarks, in otherembodiments such a server is implemented in distributed fashion. In someembodiments, one server may act as a default repository, and candispatch queries to other servers if the first server cannot provide therequested data. Caching of frequently-requested content can be providedat various locations through the network. Additional details on suchnetwork configurations can be found in application Ser. No. 09/343,104.

As is familiar to those skilled in the arts, the foregoing methods maybe performed using dedicated hardware at devices 12, 14 and 20, and/orthrough use of processors programmed in accordance with firmware orsoftware, etc. In the latter case the processors may each include a CPUand associated memory, together with appropriate input and outputdevices/facilities. The software can be resident on a physical storagemedia such as disks, and can be loaded into the processors' memory forexecution. The software includes instructions causing the CPU to performthe various processes detailed above.

To provide a comprehensive disclosure without unduly lengthening thisspecification, applicant incorporates by reference the patents andapplications cited above.

In view of the wide variety of embodiments to which the principles ofour invention can be applied, it should be recognized that the detailedembodiments are illustrative only and should not be taken as limitingthe scope of the invention. Rather, we claim as our invention all suchembodiments as may come within the scope and spirit of the followingclaims, and equivalents thereto.

1. A method for obtaining a high resolution output, the methodcomprising: receiving a relatively low fidelity media signal;extracting, using a processor, identifying data from audio or videoportions of the relatively low fidelity media signal; locating, based onthe identifying data, a higher fidelity media signal, relative to therelatively low fidelity media signal, on a network; and receiving thehigher fidelity media signal.
 2. A device comprising: an inputconfigured to receive a relatively low fidelity media signal; and aprocessor configured to: extract identifying data from audio or videoportions of the relatively low fidelity media signal; locate a higherfidelity media signal, based on the identifying data, relative to therelatively low fidelity media signal, on a network; and receive thehigher fidelity media signal.
 3. A computer readable medium havinginstructions stored thereon, that if executed by a computing devicecause the computing device to perform operations comprising: receiving arelatively low fidelity media signal; extracting identifying data fromaudio or video portions of the relatively low fidelity media signal;locating a higher fidelity media signal, based on the identifying data,relative to the relatively low fidelity media signal, on a network; andreceiving the higher fidelity media signal.
 4. A method for obtaining apristine output, the method comprising: receiving relatively low qualitycontent; extracting identifying data, using a processor, from audioportions or video portions of the relatively low quality content;locating, based on the identifying data, relatively pristine contentrelated to the relatively low quality content on a network; andreceiving the relatively pristine content.
 5. A device comprising: aninput configured to receive a relatively low quality content; and aprocessor configured to: extract identifying data from audio or videoportions of the relatively low quality content; locate, based on theidentifying data, relatively pristine content related to the relativelylow quality content on a network; and receive the relatively pristinecontent.
 6. A computer readable medium having instructions storedthereon, that if executed by a computing device cause the computingdevice to perform operations comprising: receiving relatively lowquality content; extracting identifying data from audio portions orvideo portions of the relatively low quality content; locating, based onthe identifying data, relatively pristine content related to therelatively low quality content on a network; and receiving therelatively pristine content.
 7. A method comprising: receivingidentifying data and information from a handheld device, wherein theidentifying data has been calculated from audible portions or videoportions of first content utilizing at least one processor housed withinthe handheld device, wherein the handheld device comprises a mediaplayer for rendering audio or video content to a user of the handhelddevice, and wherein the information comprises capabilities associatedwith the handheld device; determining, with reference to a database,additional content, enhanced or improved relative to the first content,wherein the additional content is associated with the identifying dataand the information; and providing metadata associated with theadditional content to the handheld device.
 8. The method of claim 7,wherein the handheld device comprises a cell phone.
 9. The method ofclaim 7, wherein the identifying data is calculated by decoding adigital watermark hidden in data representing the first content.
 10. Themethod of claim 7, wherein the information indicates media playcapabilities.
 11. A computer readable medium having instructions storedthereon, that if executed by a computing device cause the computingdevice to perform operations comprising: receiving identifying data andinformation from a handheld device, wherein the identifying data hasbeen calculated from audible portions or video portions of first contentutilizing at least one processor housed within the handheld device,wherein the handheld device comprises a media player for rendering audioor video content to a user of the handheld device, and wherein theinformation comprises capabilities associated with the handheld device;determining, with reference to a database, additional content, enhancedor improved relative to the first content, wherein the additionalcontent is associated with the identifying data and the information; andproviding metadata associated with the additional content to thehandheld device.
 12. An apparatus comprising: an input configured toreceive identifying data and information from a handheld device, whereinthe identifying data has been calculated from audible portions or videoportions of first content utilizing at least one processor housed withinthe handheld device, wherein the handheld device comprises a mediaplayer for rendering audio or video content to a user of the handhelddevice, and wherein the information comprises capabilities associatedwith the handheld device; and an electronic processor configured to:determine additional content, enhanced or improved relative to the firstcontent, wherein the additonal content is associated with: i) theidentifying data, and ii) the information; and provide metadataassociated with the additional content to the handheld device.
 13. Theapparatus of claim 12, wherein the handheld device comprises a cellphone.
 14. The apparatus of claim 12, wherein the identifying data iscalculated by decoding a digital watermark hidden in data representingthe first content.
 15. The apparatus of claim 12, wherein theinformation indicates media play capabilities.